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ABSTRACT 



New York Networks for School Renewal (NYNSR) is a 5-year 
collaborative project begun in 1995 as part of an effort to revitalize U.S. 
schools through public-private partnerships. Four New York organizations with 
years of experience in public education reform have joined in the NYNSR 
collaboration. An outcomes evaluation collected and analyzed both 
school-level and student-level data from NYNSR schools using databases 
constructed for the program. The NYNSR began with 80 founding schools and 
added an additional 60 public schools and programs, many in low- income areas. 
These 140 schools, which have a larger population of African American and 
Latino students than other New York City public schools, serve some 50,000 
students. Between spring 1996 and spring 1997, the proportion of students in 
the 80 "founding schools" who read at or above the national norms for grades 
3 through 8 rose 5%. More than 70% of parents and guardians surveyed were 
satisfied or very satisfied with the quality of teaching, what students 
learn, and safety in the schools. Although the smaller schools had a somewhat 
higher cost per student, their higher graduation rates and lower dropout 
rates mean that they have produced the lowest cost per graduate in the city 
school system. In addition to increasing student achievement, NYNSR has 
expanded community and outside institutional involvement in the creation, 
governance, and culture of public schools. (Contains 6 tables, 5 figures, 21 
graphs, and 1 map.) (SLD) 
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L INTRODUCTION 



Evaluation Design, Analytic Issues 

The outcomes evaluation of the NYNSR project, conducted by NYU’s Institute for 
Education and Social Policy, seeks to answer two basic questions: who are the students in 
NYNSR schools, and how well are they doing in terms of academic performance? 

Our evaluation design answers the first question two ways: by collecting, aggregating and 
analyzing school-level data about NYNSR schools; and by collecting, aggregating and 
analyzing student-level data for the students in NYNSR schools. Who We Are , the first 
report of the outcomes study, published by NYNSR in 1997, analyzed student demographics 
from the 1995-96 school-level data for NYNSR schools. This Progress Report analyzes 
some additional school-level data for the 1995-96 and 1996-97 school years, but 
concentrates mosdy on new student-level data for NYNSR students and schools for 1995-96 
and 1996-97. Most of our findings in this report confirm the earlier results reported in Who 
We Are . But we also offer some new findings, and correct a few findings reported in Who 
We Are , by using the greater precision that student-level data offer. 

The second question our evaluation design asks — how well are the students in NYNSR 
schools doing — will be answered through a series of comparisons. One set of longitudinal 
comparisons is based on following students in the 80 founding NYNSR schools across time, 
using student-level data from the 1 995-96, 1 996-97 and 1 997-98 school years. To compare 
the performance of these NYNSR students across time, we will examine patterns of 
attendance; enrollment consistency or its converse, mobility; graduation or dropout in high 
schools; and the year-to-year gain or loss in test score results. 

Another set of comparisons is between particular groups of NYNSR students and students 
in non-NYNSR schools selected for their demographic similarities. We have matched many 
of the NYNSR founding schools with non-NYNSR New York City public schools on a 
variety of student demographic and outcome characteristics. We will compare what we call 
our Panel 1 students (students who were in NYNSR founding schools in 1995-96) with 
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students in our matched comparison sample, across a range of demographic and 
performance indicators, to answer the question of how well NYNSR students are doing. We 
will record the year-to-year changes in Panel 1 students as they move from 1995-96 to 1996- 
97 and 1997-98, and compare those year-to-year changes with the changes for students in 
the comparison pool. 

To provide yet another assessment of how well NYNSR students are doing, we will compare 
our panel of students, on all demographic and outcome measures, and for year-to-year 
changes, to students from a random sample drawn from all non-NYNSR New York City 
public schools. 



Figure 1 



Comparison for the NYNSR eva 


nation 




1995-96 


1996-97 


1997-98 


NYNSR 
Students 
Panel 1 


Baseline Cohort File 


Year 1 Cohort File 


Year 2 Cohort File 


Students in 
Matched 
Schools 


Baseline Cohort File 


Year 1 Cohort File 


Year 2 Cohort File 


Students in 
Random Sample 
Schools 


Baseline Cohort File 


Year 1 Cohort File 


Year 2 Cohort File 



Our evaluation design is structured, through all these comparative analyses, to allow us to 
say; students in NYNSR schools have made X amount of progress on all these indicators, 
and this is more or less progress than students in other similar New York City public schools 
have made. This Progress Report uses only 1995-96 and 1996-97 student-level data to 
analyze within-year and year-to-year student mobility in NYNSR founding schools, and also 
reports pr eliminar y outcomes on reading test scores for the same students. We present 
findings only for NYNSR students in founding schools in 1995-96 and 1996-97; our next 
progress report will present comparative findings using our comparison and random samples 
of students in the rest of the NYC system. 
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Status of NYNSR School-level Databases 



As of December, 1998, lESP has recorded in Board of Education (BOE) databases 130 
distinct NYNSR school units, some of which are small schools sharing a building with other 
NYNSR or non-NYNSR schools. (Another nine schools were slated to open during the 
1998-99 school year.) We continue to struggle with multiple BOE school identifiers for 
schools with IC-8 or 6-12 grade configurations^ thus, for example, the School for the Physical 
City might be identified as a 6-8 middle school in District 2’s aggregate data, and as a 9-12 
high school in the Manhattan Borough Superintendency’s aggregate data. 

The school-level data we use in this report is for the 1995-96 and 1996-97 school years. 
Student- and school-level data for the 1997-98 school year is expected to become available 
d urin g the first quarter of 1999. When we receive the high school data from the 1997-98 
BOE cohort report, we will update the graduation and dropout analysis of founding 
NYNSR high schools (see Attachment A of the July 98 Progress Report). 

Status of NYNSR Student-level Databases 

At this point, we have constructed student demographics and outcome databases for each of 
the 130 NYNSR schools we have identified, from student-level data provided by the BOE, 
for the first two years of the NYNSR project, 1995-96 and 1996-97. When, later in this 
report, we describe the demographics of the students served by the project, as well as their 
mobility and their reading progress, we are reporting from data that covers up to the Spring 
of 1997. We have not yet received Math test data for any students for any year. 

The following diagram indicates which student-level data we have obtained from the BOE, 
and which data we are awaiting. 




8 



3 



Figute 2 

Status of Databases for the NYNSR evaluation 





1995-96 


1996-97 


1997-98 


NYNSR 

Students 


Baseline Cohort File 
(obtained July 98) 
N=24,468 


Year 1 Cohort File 
(obtained Dec 98) 
N=52,649 


Year 2 Cohort File 

Not Yet Obtained 
from BOE 


Students 

in 

Matched 


Baseline Cohort File 

Received 2/99 
Not yet analyzed 


Year 1 Cohort File 
(obtained Dec 98) 
N=53704 


Year 2 Cohort File 

Not Yet Obtained 
from BOE 


Students in 
Random Sample 
Schools 


Baseline Cohort File 

Received 2/99 
Not yet analyzed 


Year 1 Cohort File 
(obtained Dec 98) 
N=72219 


Year 2 Cohort File 

Not Yet Obtained 
from BOE 



lESP completed its nonrandom and random comparison sampling strategies during the 
Sli mm er of 1998, and submitted its listing of comparison schools for which we needed 
student-level data to the BOE. By February 1999, lESP had received 1995-96 and 1996-97 
data from the BOE for 125,923 students in the comparison schools, both non-random and 
random. But because we have only just begun to analyze this data, aggregate comparisons 
between NYNSR and non-NYNSR students will be presented in the next progress report. 
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II. THE STATE OF THE NYNSR PROJECT 



NYNSR Schools by Sponsoring Organization 

Table 2a presents participating NYNSR schools by sponsoring organization. Throughout 
the first three years of the evaluation, New Visions has brought new schools into the project 
by providing plan ning grants for groups seeking to start new schools. Almost half of the 
founding schools' were sponsored by the Center for Collaborative Education (CCE). The 
Center for Educational Innovation's strategy involves supporting large existing elementary 
schools as they restructure into smaller, more effective units. 



Table 2a 

Total number of NYNSR schools 
By Sponsor and Project Year 





1995-96 


1996-97 


1997-98 


Project Total 
as of June 98 


ACORN 


2 


1 


0 


3 


Center for Collaborative Education 


37 


1 


0 


38 


Center for Education Innovation (CEI) 


19 


27 


0 


46 


New Visions (NV) 


19 


10 


11 


40 


jointly sponsored (NV, CCE) 


3 


0 


0 


3 


Total 


80 


39 


11 


130 



By the end of the 1997-98 academic year, we have identified 130 public schools affiliated 
with the NYNSR initiative in New York City. Another nine NYNSR schools were slated to 
open for the 1998-99 school year. The data in Section III of this report focuses only on the 
119 NYNSR schools serving students during the 1995-96 (80 schools) and 1996-97 (another 
39 schools) school years.^ 

NYNSR School Configurations 

Table 2b presents NYNSR schools by school level and project year. In 1995-96, almost half 
of all NYNSR founding students were high school students. A large group of NYNSR 



1 “Founding schools” refers to NYNSR schools that joined the project at the start of the 1995-96 school year. 

^ One founding school sponsored by ACORN, The Rockaway New School, closed after the 1995-96 year. Since this 
school did not have a school identifier or BDS code, their data has never been available for analysis. 



o 

ERIC 



10 



5 



schools (17 schools) bridge the middle school-high school transition and serve students from 
6* to 12* grade. 



Table 2b 
NYNSR Schools 



By Project Year and Grade Configu ration 





1995-96 


1996-97 


1997-98 


Project Total 
to date 


Lower grades only (ES) 


20 


16 


2 


38 


Lower and middle 


1 


5 


0 


6 


Middle grades only (JHS or IS) 


15 


12 


4 


31 


Middle and high grades 


16 


1 


0 


17 


High grades only (HS) 


27 


5 


5 


37 


K42 


1 


0 


0 


1 


Total 


80 


39 


11 


130 



As Table 2c indicates, more than one quarter of the NYNSR project students attended 
elementary schools in 1995-96, but this proportion increased to over one third in 1996-97. 
Over the past two years, CEI and New Visions have sponsored more elementary and middle 
schools, so now the project has a slightly higher proportion of students in the early grades. 



Table 2c 
NYNSR Students 

By School level 



Grades served 


1995-96 


1996-97 


Number 


Percent 


Number 


Percent 


Elementary 


6679 


27.3 


18652 


35.3 


Middle 


6290 


25.7 


17454 


33.2 


High 


11499 


47.0 


16632 


31.6 


Total 


24468 


100.0 


52468 


100.0 



Geographic Distribution of NYNSR Schools Across New York City 

The map on the next page shows the geographic distribution of NYNSR schools as of June 
1997. The project is still heavily centered in Manhattan, but there has been a substantial 
expansion into the Bronx and other boroughs in the 1996-97 year. 

The bar charts on the page following the map show the distribution of elementary, middle 
and high schools in NYC. The bar chart at the top of the page displays the distribution of 
elementary and middle schools. Districts 1, 2, 3, and 4 in Manhattan have the largest 



NYNSR Schools as of June 1997 
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Distribution of NYNSR Elementary and Middle Schools by Community School District 

as of June 1997 



Manhattan 



o 

o 

£ 

u 

Ifl 



E 



Bronx 



Brooklyn 



Chancellor’s 
Queens Dist rict 




■ Additional Participating 
Schools 

■ Founding Schools 



1 2 3 4 5 6 7 10 11 12 13 15 19 20 22 23 24 26 30 33 

School Districts 



• Districts not included in this chart did not have NYNSR schools as of June 1 997. 

•• Middle Schools in Community School Districts that are part of a larger high school (often grades six through twelve) are counted as high 
schools and thus not represented here. 



Distribution of NYNSR High Schools by Borough as of June 1997 




■ Additional Participating 
Schools 

■ Founding Schools 




Borough 
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number of NYNSR schools and Districts 1 and 2 also have the largest number of additional 
participating schools. Districts 5, 11, 13, 24 and 26 joined NYNSR in its second year with a 
number of additional participating schools. 

The bar chart on the previous page displays the distribution of NYNSR high schools by 
borough. Again, Manhattan has a larger number of schools compared to the other 
boroughs. Additional participating schools have been added to four boroughs: three in 
Manhattan, two in the Bronx, one in Brooklyn and one in Queens. 

School Size 

Most of the NYNSR schools have small student populations. The addition of a number of 
large restructuring elementary schools to the project in the 1996-97 year has raised the 
average size of a NYNSR elementary school from 356 students in 1995-96 to 506 students in 
1996-97. The average size of NYNSR schools at each school level is shown in Table 2d. 



Table 2d 

Average Registers of NYNSR and NYC schools 





[995-96, and 1996-97 




Average NYNSR 


Average NYC 




School Register 


School Register 




1995-96 


1996-97 


1995-96 


1996-97 


Elementary school 


356 


506 


762 


773 


Middle school 


351 


709 


876 


851 


High school 


326 


340 


1595 


1483 



Note: NYNSR Middle-high schools are included in the hig^ school averages. 



The average size of a NYNSR middle school also jumped considerably due to the addition 
of a number of large schools (roughly 1000-1500 students) undergoing resttucturing. The 
average size of a NYNSR elementary school would be quite a bit lower if we were able to 
calculate student registers by mini-schools. For example, our databases only recogmze the 
13 elementary school buildings that house a reported 40 mini-schools created by CEI in the 
1996-97 year. 
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New vs. Existing Schools 



Figure 5 



Length of Time NYNSR Schools Have Been in Existence 
as of June 1997 




C3NYNSR schools opening in 1993-94 and before 

■ NYNSR schools opened in 1 994-93 

□ NYNSR schools opened in 1993-96 and more 
recently 



Note: Three NYNSR schools are not represented because we cannot verify their starting date. 
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III. WHO ARE THE STUDENTS IN THE NYNSR PROJECT? 



The information in this section of the report relies on cross-sectional data for the 1995-96 
and 1996-97 school years.^ Different schools and students participated in each year. Where 
we draw comparisons with citywide demographics, we use school-level data from the Board 
of Education’s Aimual School Reports or High School Overviews. In all cases, comparison 
data here represents all NYC public school students including NYNSR students. In future 
reports, NYNSR students will be compared with samples of non-project student specific 
data drawn for the evaluation. 

Gender and Race/Ethnicity 

Tables 3a, 3b and 3c describe the gender and ethnicity of students attending NYNSR 
founding schools during the 1995-96 year. Overall, the gender distribution in NYNSR 
schools was 51.5% female in 1995-96 and 50.4% in 1996-97. NYNSR elementary and 
middle schools served almost equal numbers of males and females, but more females than 
males were attending NYNSR high schools in 1995-96 (53.9%, n— 61 15). 



Table 3a 

NYNSR Students, Percent Female 
1995-96 and 1996-97 





1995-96 


1996-97 


N 


% 


N 


% 


Percent Female 


12601 


51.5 


26558 


50.4 




i For the most part, 1995-96 data represents 80 schools, and 1996-97 data represents 119 schools. 
•* This may reflect dropout trends that show more boys than girls dropping out of high school. 
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Table 3b 

AU NYNSR Students, 1995-96 

_^^^ace/Eduuci2_and^choo^^ 





NYNSR* 


School level 


Elementary 


Middle 


High 


N 


% 


N 


% 


N 


% 


N 


% 


Hispanic 


9955 


45.4 


3531 


55.1 


2774 


46.6 


3650 


38.1 


Black (non Hispanic) 


8572 


39.1 


1717 


26.8 


2442 


41.1 


4413 


46.1 


White (non Hispanic) 


2300 


10.5 


756 


11.8 


509 


8.6 


1035 


10.8 


Asian or Other 


1112 


5.1 


410 


6.4 


223 


3.7 


479 


5.0 


Total 


21939 




6414 




5948 




9577 





*Note: 2529 (10.3%) students are missing ethnicity data. 



In 1995-96, about 84.5% of all students in NYNSR founding schools were Hispanic or Black 
(45.4% Hispanic, 39.1% Black)^ There were different racial/ethnic distributions, however, 
across the school levels. NYNSR schools with elementary grades were 55% Hispanic, 
whereas NYNSR high schools served a larger proportion of Black students (46.1%). 



Table 3c 

All NYNSR Students, 1996-97 

by Race/Ethnidty and School level 





NYNSR* 


School level 


Elementary 


Middle 


High 


N 


% 


N 


% 


N 


% 


N 


% 


Hispanic 


20040 


38.3 


8321 


45.0 


5444 


31.3 


6275 


38.2 


Black (non Hispanic) 


21141 


40.4 


5419 


29.3 


7905 


45.5 


7817 


47.6 


White (non Hispanic) 


6506 


12.4 


2661 


14.4 


2322 


13.4 


1523 


9.3 


Asian or Other 


4593 


8.8 


2092 


11.3 


1698 


9.8 


803 


4.9 


Total 


52280 


99.9 


18493 


100.0 


17369 


100.0 


16418 


100.0 



Note: Only 368 (0.7%) of all students in the 1996-97 databases are missing ethnicity data. 



In 1996-97, about 78.7% of all students in the NYNSR founding schools were Hispanic or 
Black (38.3% Hispanic, 40.4% Black). There were different racial/ethnic distributions, 
however, across school levels. Whereas NYNSR schools with elementary grades were 45% 
Hispanic, both the middle schools and high schools served a larger proportion of Black 
students than Hispanic students (45.5% vs. 31.3% in middle schools and 47.6% vs. 38.2% in 
high schools). 

5 10.3% of all students in the 1995-96 NYNSR database were missing ethnicity data. In this report, whenever we cross two 
variables, missing cases on either of the variables or both will be missing in the table. 
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Tables 3d and 3e contrast NYNSR student race/ethnicity with the race/ ethnicity of all 
students in NYC public schools by school level. 

Table 3d shows that in 1995-96, a higher proportion of NYNSR students were Hispanic at 
all school levels, although the difference is smaller at the high school level. NYNSR 
founding schools served a slighdy larger proportion of Black students when compared with 
schools in NYC at the middle school and high school levels. The proportion of Black 
students in NYNSR elementary schools was lower than in schools in NYC. 



Table 3d 

NYNSR Students vs. All NYC Public School Students, 1995-96 

By Ethnicity and School level 





Elementary 


Middle 




h 


NYNSR 


NYC 


NYNSR 


NYC 


NYNSR 


NYC 


Hispanic 


55.1 


38.3 


46.6 


37.2 


38.1 


34.8 


Black (non Hispanic) 


26.8 


35.0 


41.1 


35.5 


46.1 


37.8 


White (non Hispanic) 


11.8 


16.7 


8.6 


17.1 


10.8 


16.4 


Asian or Other 


6.4 


10.0 


3.7"1 


10.1 


5.0 


11.0 



Source for all NYC comparison data: 1995-96 Annual School Reports and High School Overviews 



Table 3e shows that in 1996-97, a higher proportion of NYNSR students were Hispanic at 
the elementary and high school levels. NYNSR founding schools served a slighdy larger 
proportion of Black students when compared with schools in NYC at the middle school and 
high school levels. The proportion of Black students in NYNSR elementary schools was 
lower than in schools in NYC. 



Table 3e 

NYNSR Students vs. All NYC Public School Students, 1996-97 
by Ethnicity and School level 





Elementary 


Middle 


High 


NYNSR 


NYC 


NYNSR 


NYC 


NYNSR 


NYC 


Hispanic 


45.0 


38.7 


31.3 


36.9 


38.2 


34.9 


Black (non Hispanic) 


29.3 


34.9 


45.5 


35.0 


47.6 


37.0 


White (non Hispanic) 


14.4 


16.1 


13.4 


17.2 


9.3 


16.0 


Asian and Other 


11.3 


10.2 


9.8"^ 


10.6 


4.9 


12.1 


Total 


100.0 


100.0 


100.0 


100.0 


100.0 


100.0 



Note: These 1996-97 data include students in additional participating schools. Source for all NYC comparison 
data: 1996-97 Annual School Reports and High School Overviews. 



o 

ERIC 



18 



11 



Students Eligible for Free Lunch 



Eligibility for the federal free and reduced price lunch program is a very limited and 
imperfect indicator, but it is frequendy used as a proxy for student economic need or 
poverty status. Table 3f contrasts the percent of NYNSR and all NYC public school 
students eligible for free or reduced price lunches in 1995-96. Students eligible for free or 
reduced price lunches were slighdy over represented in NYNSR schools, particularly at the 
middle school level.^ 



Table 3f 

NYNSR and NYC Students 



Eligible for Free or Reduced Price Lunch 
1995-96, By School level 



Students in: 


NYNSR 


All NYC* 


N 


% 


N 


% 


Elementary schools 
Middle schools 


5275 

4761 


79.0 

75.7 


N/A 

N/A 


76.3 

69.1 



*Source: 1995-96 and 1996-97 Annual School Reports 



English Language Learners 

Overall, 12.9% of all NYNSR students (n=3166) were designated as English Language 
Learners (ELL) in 1995-96^ This percentage remained roughly the same as the project 
expanded to serve a total of 6212 (11.8%) KTX students in 1996-97. These percents are 
higher than we were able to detect with the school-level data used in our earlier report, Who 
We Are, NYNSR, 1997. We have no comparable KTX percent for the NYC public school 
system as a whole. 

Table 3g indicates that in 1995-96, NYNSR English Language Learners were fairly equally 
represented at all school levels. Of the total number of ELL students, (n=3158) 37.7% 
were in elementary school, 26.4% were in middle schools, and 35.7% were in high schools. 



6 Analysis of free lunch information for students at the high school level is not presented here due to large amounts of 
missing data. 

Although the state and city refer to the population of Limited English Proficient (LEP) students, the NYNSR project 
prefers to use the term "English Language Learner" or ELL. The LEP and ELL abbreviations are used interchangeably in 
this document. 
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Table 3g 

English Language Learners in NYNSR Schools 
By School level 





1995-96 


1996-97 


Number 


Percent 


Number 


Percent 


Elementary 


1195 


37.7 


2526 


40.7 


Middle 


835 


26.4 


2055 


33.1 


High 


1129 


35.7 


1631 


26.2 


Total 


3158 




6212 





*Note: 0.2% of ELL students (n=8) are missing level data. 
Percents may not add up to 100 due to rounding. 



NYNSR ELL Students Compared with NYC Averages 



Table 3h compares ELL students in NYNSR schools to those in NYC as a whole. Fewer 
NYNSR students are ET.T. across all school levels, and in both years. 



Table 3h 

NYNSR ELL Students vs. All NYC ELL Students 
by School level 





NYNSR 


NYC 


1995-96 


1996-97 


1995-96 


1996-97 


Elementary 


17.9 


13.6 


20.2 


18.3 


Middle 


13.3 


11.8 


16.3 


14.7 


High 


9.9l 


9.8 


15.2 


14.8 



Source for NYC comparisons: Annual School Reports and High School Overviews 



Table 3i shows that ELL students were primarily Hispanic (78.2%, n=2297). The next 
largest group of NYNSR English Language Learners were Asian/ Other (10.3%, n=302). In 
1996-97, there was a rise to 17.2% ELL for Asian students (n=1070). 



Table 3i 

English Language Learners in NYNSR Schools 



By Ethnicity 





1995-96 


1996-97 


N 


% 


N 


% 


Hisoanic 


2297 


78.2 


4275 


68.9 


Black (non Hispanic) 


158 


5.4 


437 


7 


White (non Hispanic) 


179 


6.1 


425 


6.8 


Asian or Other 


30T1 


10.3 


1070 


17.2 


Total 


2936^ 


100.0 


6212 


100.0 



*Note: 7.2% of ELL students (n=230) are missing ethnicity data. 
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special Education Students 



Overall, 3.9% (n=961) of aU NYNSR students (n=961) were designated as Special 
Education students (SE) in 1995-96.® This increased to 5.2% of all project students 
(n=2755) in 1996-97. 



Table 3j presents Special Education students by school level. Most NYNSR Special 
Education students were enrolled in middle schools (58%). Another quarter were registered 
in elementary schools (24.6%). Only 17% of NYNSR Special Education students were 
enrolled in high schools. 



Table 3j 

NYNSR Special Education Students 
by School level 





1995-96 


1996-97 


N 


% 


N 


% 


Elementary 


234 


24.3 


955 


34.7 


Middle 


564 


58.7 


1348 


48.9 


High 


163 


17.0 


452 


16.4 


Total 


96l1 


100.0 


2755 


100.0 



Note: These numbers do not include District 75 students or Pre- 
Kindergarten students. 

Source: Annual School Reports and High School Overviews 



Table 3k presents data on the ethnicity of Special Education students in NYNSR schools. 
In 1995-96, Black and Hispanic students together comprised 95.3% of the NYNSR Special 
Education student population (56.1% and 39.2%). In 1996-97, the combined percentage 
was 91%, a slight decline in the proportion of Hispanics and Blacks among Special 
Education students in NYNSR. 



8 These figures are for full-time Special Education students only and do not include students receiving resource room or 
consultant services. 
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Table 3k 

Special Education Students in NYNSR Schools 





1995-96 


1996-97 


N 


% 


N 


% 


Hispanic 


507 


56.1 


1253 


45.5 


Black (non Hispanic) 


354 


39.2 


1252 


45.4 


White (non Hispanic) 


33 


3.7 


169 


6.1 


Asian or Other 


9 


1.0 


81 


2.9 


Total 


903 


100.0 


2755 


99.9 



♦Note: 6.1% of SE students (n=58) are missing ethnicity data. 



A closer look at NYNSR Special Education students by school level in comparison to NYC 
Special Education students (Table 31) suggests that, in both years, a greater proportion of 
NYNSR middle school students out of the entire NYNSR student population were 
categorized as Special Education students (1995-96, 8.8%; 1996-97, 7.5%). 



Table 31 

NYNSR SE Students vs. All NYC SE Students 

by School level 





NY 


NSR 


K! 


iC 


1995-96 


1996-97 


1995-96 


1996-97 


N 


% 


N 


% 


N 


% 


N 


% 


Elementary 


234 


3.5 


955 


5.1 


N/A 


6.9 


N/A 


5.8 


Middle 


564 


9.0 


1348 


7.7 


N/A 


7.5 


N/A 


7.5 


JM 


163 


1.4 


452 


2.7 


N/A 


6.3 


N/A 


5.5 



Note: 1996-97 NYNSR data includes additional participating schools. 
Source: Annual School Reports and High School Overviews. 



In 1996-97, NYNSR schools served roughly the same percentages of elementary and middle 
school students in special education as were served in all NYC schools. The percentage of 
high school students in special education served by NYNSR high schools was roughly half 
the percent served by all NYC schools. 
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Students Overage for Grade 



Table 3m 



Students Years Overage for Grade 
1995-96 





NYNSR High 
Schools 


NYC High 
Schools 


N 


% 


N 


% 


All 9^ or 10^ graders entering Articvdated 
Alternative high schools 


2312 


42,3 


20902 


33,0 



*Note: “Overage” applies only to General Education students. Only students entering traditionally 
structured (grades 9-12) high schools in Grade 9 are included in NYNSR figures. Students in 
alternative transfer high schools, and retained students are not included in figures. Source for NYC 
comparison data: 1995-96 and 1996-97 High School Overview Reports: Profile of the Entering Class. 



Overage for grade means that a student is more than one year older than his or her normal 
grade-level age should be. A large proportion of overage students suggests that these 
students have been retained earlier in their education. This table indicates that more 
NYNSR 9^ and 10^ grade students were overage for grade in 1995-96 than were their 
counterparts in similar settings in the NYC high school system. 
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IV. STUDENT MOBILITY 



The effect of student mobility on instruction and subsequent student performance is a 
concern in New York City. A 1992 study by the New York State Education Department 
foimd high student mobility rates in New York City were consistendy related to low student 
performance on state mandated tests.® 

Student mobility is an important variable in our analyses for two reasons. First, substantial 
student movement dviring the school year often affects classroom environments, makes 
instruction more difficult, and lowers student performance. Moreover, large numbers of 
students changing schools between any two academic years can reduce the validity of cross- 
sectional analyses of reading performance data used to indicate school-level achievement, if 
different students are represented in the school data for each year. The following analyses 
explore levels of student mobility in NYNSR and NYC public schools. Comparable analysis 
for non-NYNSR students will be presented in the next progress report. 

Within-Year Mobility In NYNSR Founding Schools 

To estimate the amount of student movement in NYNSR schools during the 1995-96 year, 
we constructed a baseline cohort file comprising all students enrolled in all founding 
NYNSR schools at any of the following points in time; October 1995, March 1996, or June 
1996. NYNSR students changing schools within the 1995-96 year were assigned to a 'home 
NYNSR school' for all analyses. 

What we found is important: more than 94% of all founding elementary and middle school 
students remained in the same NYNSR school during the year, whereas approximately 91% 
of all NYC public elementary and middle school students remained in the same public 
school d urin g the year. Table 4a presents the percent of NYNSR students remaining in the 
same school between October 1995 and June 1996, compared to aU NYC students. 

5 Student and Teacher MobUity: Impaa on School Performance in New York City pubUc schools. New York State Education 
Department, Albany. Office for Planning, Research and Support Services. 1992. 

>» We assigned a ‘home school’ for the 1995-96 year by examining the student’s school location for three separate points in time: 
October 1995, March 1996, and June 1996. If two points in time (or more) were available and the same, we assigned the student 
to that school.’ If the schools attended differed, and/or some data were missing, we assigned the student to the last school 
attended during the 1995-96 year. 
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Table 4a 

Percent of NYNSR and NYC students in the same school 
Between Oct 95 and June 96 





NYNSR 


NYC 


All elementary schools 


94.2% 


91.2% 


All middle schools 


94.4% 


91.5% 



*Note: Source for NYC comparison data is 1995-96 Annual School Reports. 



Table 4b 



Within-year Mobility between project and non-project schools 
NYNSR founding students 

^ of June 1996 





Number 


Percent 


Students remaining in the project throughout 1995-96 


22440 


91.7 


Students moving into NYNSR schools 


992 


4.1 


Students moving out of NYNSR schools 


1036 


4.2 


Total 


24468 


100.0 



*Note: More than three changes of schools could not be detected in our analyses. 



When students did move between schools during the 1995-96 year, equal proportions of the 
students moved to and from the NYNSR project, as seen in Table 4b. There does not 
appear to be a large influx or exodus from founding project schools over the course of the 
1995-96 school year. 

Persisters and Leavers: Annual Student Mobility 

It has long been obvious that students who continue their education produce better 
academic outcomes than students who interrupt or terminate their education short of 
graduation. The research literature on schooling outcomes has established that schooling 
persisters produce better lifetime outcomes, particularly in terms of aggregate earnings, than 
school leavers. The following table presents the proportion of students in founding NYNSR 
schools in 1995-96 who remained in the NYC public school system a year later. 



O 
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Table 4c 

Who Stays and Who Leaves? 

Status of founding 1995-96 NYNSR Students 





Number 


Percent 


Persisters 


20221 


90.2 


Leavers 


2187 


8.9 


Total 


22408 


99.9 



Note: This table does not include Pre-K students, students who 
began the year in Special Education, or students missing level 
data in either of the two years. 



Table 4c reports that 90.2% (n=20,221) of the NYNSR founding students persisted in NYC 
public schools a full year after the project began. "Persisters" includes students who were 
promoted to the next grade, graduated from high school, were retained on grade, or were 
demoted to the prior grade, as well as students who were referred and transferred to self- 
contained Special Education classes or GED programs. "Leavers" includes students who 
transferred out of the NYC public schools, students who dropped out of school, or students 
whose reasons for leaving are unknown as of June 1997. Comparable data for non-NYNSR 
smdents was not yet available at the time we released this report. 



Grade Promotion/ Grade Retention 

Table 4d presents the proportion of founding NYNSR smdents who were promoted or 
retained on grade between 1995-96 and 1996-97. Approximately seventy-two percent 
(72.4%, 16,223 smdents) of the NYNSR smdents in founding schools were promoted to the 
next grade between these two years. 



Table 4d 



Grade Promotion and Retention in NYNSR Schools 
Stams of Founding NYNSR Smdents 
^ of Time 1997 





Number 


Percent 


Promoted 


16233 


72.4 


Retained 


3463 


15.5 


Other 


2712 


12.1 


Total 


22408 


100.0 


Note: "Other" includes students transferred to SPE, GED or non-NYC 



public school programs, dropped out, or unknown reason for discharge. 
This table does not include Pre-K students, students who began the year 
as a Special Education student, or students missing data in either of the 
two years. 
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V. Preliminary Outcome Analysis of Founding NYNSR Students 



Each year, all New York Qty public school students in grades 3 through 8 take a 
standardized reading test. A variety of other reading or language assessments are 
administered to elementary and middle school students, including the Performance 
Assessment in Language Arts (PAL) exam in Grade 4. A number of NYNSR schools have 
implemented sophisticated portfolio requirements in addition to the required standardized 
tests. 



This report is limited to a preliminary analysis of standardized reading test data because these 
are the only data available for both NYNSR and non-project comparison schools. Despite 
their limitations, these data are regularly used by administrators to make important decisions 
about individual schools including whether they are put on registration review or sanctioned 
in other ways. Parents and the public have learned to use the standardized test data to press 
for additional support for, or structural changes in their schools. The following analysis is 
exploratory, and future reports will present additional longitudinal work. 

Methodology 

The NYC public schools regularly report on student reading performance, and their reports 
are almost always aggregated to the school- or grade- level. The student outcomes data used 
for all analyses in this section are based on the CTB Reading Test (CTB-R) administered 
every Spring to NYC public school students in Grades 3 through 8. " Comparable analysis 
for non-NYNSR students will be presented in the next progress report.’^ 



This discussion is based on the test scores of 6,461 foimding NYNSR students in general 
education who were tested on the CTB in Spring 1996 and again in the Spring of 1997 in 



" The CTB Reading Test (CTB-R), published by CTB/McGraw-Hill is a norm-referenced assessment system The CTB-R 
consists of two English Language Arts tests. One is a multiple-choice test of reading comprehension, called CTB fading. 
The other is called the Perfonnance Assessment in Language Arts (PAL), in which students write their own responses to the 
questions. The test data for this evaluation utilize only the multiple-choice component of the CTB-R. 

'2 Beginning in 1986, the dtywide assessment system in New York City for English Language Arts was the Degrees of 
Reading Power (D.R.P.). For ten years, this test served as the single-component dtywide exam for New York City Public 
schools. The New York City Public schools system switched to the CTB-R from the Degrees of Reading Power or DRP 
after the Spring of 1995, to improve the alignment between dtywide assessment and the new, higher-order learning 
standards established by both the state and the dty system. 
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NYNSR founding schools. We obtained test data for the selected students by taking the 
list of 1995-96 founding student IDs and matching them to each year's test files, thereby 
obtaining testing data for each student regardless of whether the student moved to a new 
school (or out of the NYNSR project) in the subsequent year. (These students may 
represent a selected group if students who leave NYNSR schools differ from those who 
remain.’'') This strategy attempts to avoid biases that result from selecting only students who 
remain in NYNSR for two consecutive years. 

In this part of the NYNSR progress report, we measure student achievement gains on 
several levels. We begin by measuring whole project gains and then compare and contrast 
the gains of students of different ethnic groups. Next, we examine gains of particular 
students in each grade level (e.g. students who were 5* graders in 1995-96 and their test 
performance in the subsequent year). 



U All limited English proficient students who have been in an English language school system for less than five ye^s AND 
who scored below the 30't' percentile on the English Reading sub-test of the LAB are exempt from testmg m all subjects 
areas, and thus are not represented in the data in this report. 

*** An analysis of attrition from NYNSR schools is planned when 1997-98 data becomes available. 
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Gains in Reading 



The charts below indicate that between Spring 1996 and Spring 1997, the percent of 
NYNSR students reading at or above the national norms increased from 36% to 41%, an 
increase of 5%. 

Percent of NYNSR Students Reading at or above National Norms 
Spring 1996 and Spring 1997 



Percent At or Above National Norms 

60 < 

50 - 

40 - 

^ 30 - 
0 ) 

Q. 

20 
10 
0 

1996 1997 

Ni996 = 8242 Ni997 = 6461 




The problem with using the statistic “percent of students at or above national norms is that 
it is insensitive to any improvement in performance of a subgroup of students far from the 
national norms. An intervention that leads to a significant improvement in the learning of 
the lowest percentile of smdents, for example, can go undetected since it will not change the 
percent above national norms. An important paper by Tony Bryk and colleagues. 
Academic Productivity of Chicago Public Elementary Schools,’^ uses percentile ranges 
to demonstrate the shortcomings of using only “the percent of smdents scoring at or above 
national norms” to measure improvements in smdent achievement. Bryk recommends that 
we use mean improvement in concert with a value-added technique to reflect the 
improvement in reading of all the smdents (the lowest percentile as well as the highest). 



15 Biyif et al, Academic Productivity of Chicago Public Elementary Schools, A Technical Report Sponsored by the 
Consortium on Chicago School Research, Chicago, March 1998. 
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Quartile Analysis 



A more detailed analysis of the same NYNSR test score results reveals that the lowest 
scoring student group, the lowest quartile, experienced the greatest gains between 1996 and 
1997. In 1996, 34% of the NYNSR students ranked in the lowest quartile nationally. One 
year later, only 28% of all founding students in NYNSR schools remained in the lowest 
quartile.’^ 



Quartile Analysis, NYNSR Students, 1996 and 1997 



Initial Distribution of Students Scores, 1996 
Alt Schools 




Percent Above National 
Norms: 36.7% 

Median Percentile: 38% 
Mean Percentile: 40.1% 



Quartiles 



Distribution of Students Scores, 1997 



All Schools 
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At National Norms 




Percent Above National 
Norms: 41% 

Median Percentile: 42% 
Mean Percentile: 44.1% 
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Table 6a 

Distribution of Student Scores, NYNSR Students 





1995-96 


1996-97 


N 


% 


N 


% 


First Quartile (0-24) 


2780 


33.7 


1839 


28.5 


Second Quartile (25-49) 


2434 


29.5 


1985 


30.7 


Third Quartile (50-74) 


1677 


20.3 


1494 


23.1 


Fourth Quartile (>=75) 


1351 


16.4 


1143 


17.7 


Total 


8242 


99.9 


6461 


100.0 



Reading Analysis by Ethnicity 

Table 6b indicates that the percent of students reading at or above the national norms 
increased in every ethnic group from 1995-96 to 1996-97. After one year in a NYNSR 
school, the group experiencing the greatest change was the Hispamc students, gaining 6% 
between 1996 and 1997. Note that examining the means or medians alone would lead to the 
conclusion that all ethnic groups benefited to about the same extent from different starting 
points. All groups moved up from 2.7%-4.0% points in the mean and about 5%-6% in the 
median. 



Table 6b 

Reading Performance Statistics by Ethnicity, NYNSR Students 





Percent above National 
Norms 


Median 


Mean 


1996 


1997 


Change 


1996 


1997 


Change 


1996 


1997 


Change 


Asian and other 


54 


58 


+4 


51 


56 


+5 


51.2 


54.9 


+2.7 


Hispanic 


29 


35 


+6 


32 


38 


+6 


35.8 


40.1 


+4.3 


Black 


35 


37 


+2 


36 


41 


+5 


38.3 


42.4 


+4.1 


White 


72 


73 


+1 


63 ' 


69 


+6 


61.4 


65.3 


+3.9 
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Pereent in Category Percent in Category Percent in Category Percent in Category 



Quardle Analysis by £thnicity, NYNSR Students 1996 and 1997 
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Initial Distribution of Students Scores, 1996 
Asian or other 




Initial Distribution of Students Scores. 1996 
Hispanic 




Initial Distribution of Students Scores, 1996 

Black - Not Hispanic 
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Initial Distribution of Students Scores, 1996 
White - Not Hispanic 




Distribution of Students Scores, 1 997 
Asian or other 




Distribution of Students Scores, 1997 
Hispanic 




25-49 50-74 

Quartiles 



Initial Distribution of Students Scores, 1997 

Black -- Not Hispanic 




0-24 25-49 60-74 

Quartiles 

Distribution of Students Scores, 1997 
White - Not Hispanic 




Quartiles 



Quartiles 










The charts illustrate the differences among ethnic groups in terms of their gains after one 
year in a NYNSR school. Overall, all ethnic groups had fewer students in the lowest quartile 
in 1997 than they did in 1996. Students with the poorest performance appear to have 
benefited the most after attending a NYNSR school. There was no change from 1996 to 
1997 in the percentage of students scoring in the 25-49 quartile nationally. In all probability, 
some students in the 25-49 quartile moved above the national average in percentages roughly 
equal to the number of students that moved up from the lowest quartile. 

The charts show that the students in Asian and Other group who ranked in the lowest 
quartile nationally dropped from 21% to 15%, while the percentage above the national 
norms increased by 4%. The percent of Hispanic students that ranked in the lowest quartile 
dropped from 40% to 32%, while the percentage above the national norms increased by 6%. 
The percent of Black students who ranked in the lowest quartile dropped from 34% to 30% 
while the percentage above the national norms increased by 2%. Fewer White students 
ranked in the lowest quartile than in any other quartile, with the percentage in the lowest 
quartile decreasing from 12% to 10%. The percentage above the national norms for White 
students, which was highest among the four groups, had the lowest increase of 1% (from 
72% to 73%). 
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Grade by Grade Reading Analysis 

3"* Graders from 1996 to 1997 

In Spring 1996, 39% of the 3“* graders ranked above the national norms. In Spring 1997, the 
percent ranking above the national norms went up to 46%, an increase of 7%. In 1996, 39% 
of the graders ranked in the lowest quartile nationally, while in 1997, only 23% of the 
cohort were in the lowest quartile nationally. Clearly, project students in the lowest quartile 
are doing better. 



Quartile Analysis, NYNSR Students 1996 and 1997 




Distribution of Same Students Scores, 1997 



0-24 25-49 60-74 >=75 



Percent Above National 
Norms: 46% 

Median Percentile: 48% 
Mean Percentile: 47.8% 




Quartiles 
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4* Graders from 1996 to 1997 



In 1996, yi.1% of the 4* graders ranked above the national norms. In 1997, the percent 
ranking above the national norms went up to 38.9%, a slight increase. 



Quartile Analysis, NYNSR Students 1996 and 1997 



Initial Distribution of Students Scores, 1996 
4th Grade 




Percent Above National 
Norms: 37.7% 

Median Percentile: 41% 
Mean Percentile: 43.3% 



Quartiles 



Distribution of Students Scores, 1997 
5th Grade 




Percent Above National 
Norms: 38.9% 

Median Percentile: 42% 
Mean Percentile: 43.1% 



Quartiles 
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5* Graders from 1996 to 1997 



In Spring 1996, 34.9% of the 5* graders ranked above the national norms. In Spring 1997, 
the percent ranking above the national norms went up to 37.6%, an increase of 2.7%. 

33 . 8% of the 5* graders ranked in the lowest quartile nationally in 1996. In 1997, 35.6% of 
this cohort were in the lowest quartile nationally, a 1 .8% increase in the lowest 



Quartile Analysis, NYNSR Students 1996 and 1997 



Initial Distribution of Students Scores, 1996 
5th Grade 
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At National Norms 




Percent Above National 
Norms: 34.9% 

Median Percentile: 37% 
Mean Percentile: 39.9% 



Quartiles 



Distribution of Students Scores, 1997 
6th Grade 




Percent Above National 
Norms: 37.6% 

Median Percentile: 37% 
Mean Percentile: 41 .9% 



Quartiles 
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6“* Graders from 1996 to 1997 



In Spring 1996, 29.9% of the 6"^ graders ranked in the lowest quartile nationally. In Spring 
1997, 29.1% of this cohort were in the lowest quartile nationally, an improvement of 0.8%. 
In 1996, 31.4% of the 6^ graders ranked above the national norms. In 1997, the percent 
ranking above the national norms went up to 41.4%, an increase of 10%. Clearly the lowest 
quartile has not been affected; the improvement in reading seems to occur among those 
students in 25-49 quartile. In 1996, 38.7% of the 6^ graders ranked in the 25-49 quartile 
nationally, whereas in 1997, only 29.5% were in this quartile. 

Quartile Analysis, NYNSR Students 1996 and 1997 



Initial Distribution of Students Scores, 1996 




Quartlles 



Percent Above National 
Norms: 31.4% 

Median Percentile: 40% 
Mean Percentile: 40.1% 



Distribution of Students Scores, 1 997 
7th Graders 




Percent Above National 
Norms; 41 .4% 

Median Percentile: 42% 
Mean Percentile: 43.6% 
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VI. CONCLUSION 



The data and findings provided in this January 1999 Progress Report considerably advance 
the discussion begun in Who We Ate (NYNSR, 1997). We used school-level data to analyze 
the student and school demographics reported in Who We Are, and to demonstrate that 
NYNSR schools had higher percentages of students of color, and higher percentages of low- 
income students, than schools in the rest of the New York City system. This report uses 
student-level data to confirm those findings, and to refine some of the findings reported in 
Who We Ate about the percentages of special education students and English Language 
Learners in NYNSR schools. 

This report also provides the first comparative findings based on student-level data. We 
compare the performance of NYNSR students in 1995-96, on indicators such as persistence 
and leaving and reading test scores, to the performance of those same NYNSR students in 
1996-97. We also compare NYNSR students to the entire New York City system on 
indicators such as within-year mobility and overage for grade. 

These initial findings suggest the power of the analytic comparisons that will become 
possible when we complete the analyses of the matched sample and random sample of New 
York City schools discussed on pages 2-3 of this report. In subsequent reports, we will 
compare NYNSR students to their counterparts in the rest of the city system on a very wide 
range of indicators, including all those presented in this report We look forward to 
providing the first round of these new comparative findings in our July 1 999 Progress 
Report . 
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